The human visual system is capable of remarkable dynamic range that enables it to detect a wide range brightness levels (˜1010) from direct sunlight to dark shadows, and doing so adaptively. On the other hand, typical digital cameras provide only 8 bits, or 256 levels of brightness. Besides this reduced numeric dynamic range, the effective dynamic range is often less than that, since the pixel values are artificially gained (by analog or digital means) to enable representation of the scene within the required numeric range. This gaining operation results in the loss of granularity and increased noise levels in the recorded images and cannot recover the intrinsic loss of dynamic range due to poor pixel sensitivity. In fact, the ongoing miniaturization in the pixel size (e.g., less than 2×2 microns) can also result in reduced sensitivity which cannot be easily compensated. Thus, the intrinsic dynamic range is limited by at least the following factors:                the reduced light sensitivity of the sensor due to limited photon conversion efficiency and non-linearity of the response functions;        miniaturization of the pixel size: since the capacitance of semiconductor is proportional to the pixel area, there is an unavoidable trade-off between the pixel size and the associated light sensitivity; and        the need for longer integration times in order to acquire better pictures in low light conditions which may result in having image areas that are under-exposed and other areas that are saturated.        
From the user perspective, enhanced dynamic range imaging can have significant impact on perception of picture quality, especially because casual users are unaware of necessary lightening requirements to achieve proper photography results. Therefore, techniques to improve the intrinsic dynamic range will be highly valuable in the context of mobile imaging.
Most image sensors use the same exposure time for all pixels. Often, this results in images being too dark in some image areas and possibly saturated in other areas. In fact, the exposure control mechanism of the camera has to statistically determine a common exposure interval (for all pixels): this is usually done by choosing a value that suitable for the majority of the pixels, thus sacrificing either those areas that are too dark or too bright, i.e., causing degradation as a reduced intrinsic dynamic range.
The most evident prior art method (e.g., Debevec and J. Malik. Recovering High Dynamic Range Radiance Maps from Photographs. In SIGGRAPH 97, August 1997) is to enhance the intrinsic dynamic range by capturing several images at multiple exposures and then combining these images in order to improve the perceived dynamic range. However, this can be done primarily for static scenes, and in this approach it is difficult to obtain reliable results due to the unavoidable registration that is needed to map the pixels from the different images. The precise image registration remains a challenging algorithmic aspect. Other approaches are also tried.
For example, (see U.S. Pat. No. 5,801,773, E. Ikeda, Image data processing apparatus for processing combined image signals in order to extend dynamic range, 1998), multiple copies of the image are taken simultaneously using beam splitters that reflect the incoming light onto different sensors that are preset to sample the light at different exposures. This approach has the advantage of not requiring online registration since the images are captured simultaneously, however it is relatively more expensive since it requires additional image sensors and a careful alignment of the optical elements.
Another approach is based on differently exposed pixels within the same CMOS sensor has been proposed by D. Yang, B. Fowler, A. El Gamal and H. Tian, “A 640×512 CMOS Image Sensor with Ultrawide Dynamic Range Floating-Point Pixel-Level ADC,” IEEE Journal of Solid State Circuits, Vol. 34, No. 12, pp. 1821-1834, December 1999. In this approach the number and timing of the exposures as well as the number of bits obtained from each pixel can be freely selected and read out. However, this requires monitoring of each image pixel in order to decide whether its exposure should be terminated or not. Additionally, the fuzzy exposure mechanism can result in incoherent noise levels across the image plane, i.e., some pixels will turn out to be noisier than the others with no reliable filtering mechanism to reduce this noise.